Discardable files

ABSTRACT

The present application includes methods and system for managing a storage device. In one implementation, a storage allocator that is present in a host or a storage device receives a request to store a file in a storage area of the storage device. The storage allocator marks the file as discardable in a file system structure associated with the storage device and updates a primary file allocation table (“FAT”) to associate a cluster chain that is allocated to the file with the file. The storage allocator additionally updates a discardable FAT or a database to reflect a physical location of the file, or may generate one or more location files that store the physical location of the file. The storage allocator then manages the storage area device based on the FAT and a discardable FAT, database, or one more location files indicating the physical location of the file.

RELATED APPLICATIONS

The present application is a continuation of PCT Application No.PCT/US09/65056, filed Nov. 19, 2009, which claims priority to IndianPatent Application No. 2238/MUM/2009, filed Sep. 25, 2009, which is acontinuation-in-part application of U.S. patent application Ser. No.12/336,089, filed Dec. 16, 2008, and which claims the benefit of U.S.Provisional Patent Application No. 61/159,034, filed Mar. 10, 2009, andthe entirety of each of these applications is hereby incorporated byreference.

FIELD OF THE INVENTION

The present invention generally relates to storage devices and morespecifically to a method and to a device for managing files in a storagedevice.

BACKGROUND

Use of non-volatile storage devices has been rapidly increasing over theyears because they are portable and they have small physical size andlarge storage capacity. Storage devices come in a variety of designs.Some storage devices are regarded as “embedded”, meaning that theycannot, and are not intended to be removed by a user from a host devicewith which they operate. Other storage devices are removable, whichmeans that the user can move them from one host device (e.g., from adigital camera) to another, or replace one storage device with another.

The digital content stored in a storage device can originate from a hostof the storage device. For example, a digital camera, an exemplary host,captures images and translates them into corresponding digital data. Thedigital camera then stores the digital data in a storage device withwhich it operates. Digital content that is stored in a storage devicemay also originate from a remote source: it can be sent to a host of thestorage device, for example, over a data network (e.g., the Internet) ora communication network (e.g., a cellular phone network), and then bedownloaded by the host to the storage device. The remote source may be,for example, a service provider or a content provider. Service providersand content providers are collectively referred to hereinafter as“publishers”.

Users of storage devices can willingly download media content andadvertisements by requesting the media content or the advertisementsfrom publishers. However, sometimes, publishers, trying to increasetheir income, send content to users without asking their permission, andsometimes even without the users being aware that such content wasdownloaded to their storage devices. Content that a publisher sends tousers without getting their consent are referred to herein as“unsolicited content”. Oftentimes, unsolicited content is intended to beconsumed by users after paying, or after committing to pay, thepublisher a fee.

By downloading unsolicited content to users' storage devices publishershope that users will eventually consume the unsolicited content for afee, thus increasing their income. Publishers storing unsolicitedcontents on storage devices without asking users' consent, hoping thatthe users will consume these contents for a fee, is a concept known inthe media publishing field as “predictive consignment”. However,unsolicited content may remain stored in a storage device without theuser of the storage device knowing of its existence or wanting toconsume it. Storing unsolicited content in a storage device reduces theavailable (i.e., free) user storage space on the storage device, whichis undesirable from the user's point of view. A user may find that thereis less space in the storage device for the user's own content (e.g., amusic file) because someone else (i.e., some publisher) has taken overpart of the storage space on the storage device, or that the user mayhave to reclaim the storage space so taken by deleting the unsolicitedcontent.

One partial solution to the problem of taking over parts of the user'sstorage space involves blocking publishers' access to the storagedevice, such as by blocking the publisher's website. This solution maybe acceptable for the users but it is problematic from the publishers'point of view because publishers will make fewer sales and lose apotential income source. Another partial solution to this probleminvolves publishing content to hosts (i.e., storing content files instorage devices of these hosts) and removing the content when it becomesirrelevant. In other words, the publisher that originated the contentremoves the stored unsolicited content from the storage device when thecontent becomes irrelevant. An unsolicited content is regarded asirrelevant if the time for its consumption has lapsed, or when there areindications that the user is not likely to consume it.

There is therefore a need to address the problem with unsolicited files.Specifically, while publishers should be allowed to pursue downloads tostorage devices of unsolicited content in the course of conducting theirbusiness, these downloads should not have a materially deterring effecton the user experience.

SUMMARY

It would, therefore, be beneficial to be able to store unsolicited filesin a storage device for as long as the storage space required toaccommodate them in the storage device is not required for user's files,and to remove unsolicited files from the storage device in order toguarantee a minimum size of free storage space for user files. Variousembodiments are designed to implement such files management, examples ofwhich are provided herein.

To address the foregoing, files stored, or files to be stored, in astorage device are marked either as non-discardable or discardable in astructure of a file system associated with the storage device. Eachmarked file has associated with it a discarding priority level. A newpublisher's file (i.e., an unsolicited file) is permitted to be storedin the storage device only if storing it in the storage device does notnarrow a storage usage safety margin, which is reserved for user files,beyond a desired margin. User files, on the other hand, are allowed tobe stored in the storage device even if their storage narrows thestorage usage safety margin beyond the desired width. However, in suchcases, the desired width of the storage usage safety margin is restoredby removing one or more discardable files from the storage device. Adiscardable file is removed from the storage device if its discardingpriority level equals or is higher (or lower, as explained herein) thana predetermined discarding threshold value.

In some implementations, a storage allocator present in a host, astorage device, or a combination of both utilizes a primary fileallocation table (FAT) and a discardable FAT, database, or one or morelocation files to store a discardable file in a storage area of thestorage device. The primary FAT stores an association between a clusterchain and the discardable file, and one of a discardable FAT, database,or one or more location files indicates a physical location of the file.Information in the discardable FAT, database, or one or more locationfiles is used to override FAT entries in the primary FAT correspondingto the discardable file. By overriding FAT entries with information inthe discardable FAT, database, or one or more location files, FAT32 filesystem checking and repair utilities see clusters associated with thediscardable file as allocated rather than as data fragments (also knownas orphan clusters), thereby preventing the utilities from turning adiscardable file into a non-discardable file. The storage allocatormanages the storage area of the storage device in accordance with theprimary FAT and the discardable FAT, database, or one or more locationfiles.

The discardable file system additionally provides the ability to controlwhat operations applications may perform associated with discardablefiles based on a user ID associated with the applications. The user IDmay be an owner user ID that identifies the application or user thatcreated a discardable file. Typically, an application associated withthe owner user ID is provided the ability to define what applicationsassociated with additional user IDs may access the discardable file andwhat actions applications associated with the additional user IDs maytake with respect to the discardable file. An additional user ID may beassociated with a single application or a single user, or the additionaluser ID may be a shared user ID that is associated with multipleapplications or multiple users.

BRIEF DESCRIPTION OF THE DRAWINGS

Various exemplary embodiments are illustrated in the accompanyingfigures with the intent that these examples not be restrictive. It willbe appreciated that for simplicity and clarity of the illustration,elements shown in the figures referenced below are not necessarily drawnto scale. Also, where considered appropriate, reference numerals may berepeated among the figures to indicate like, corresponding or analogouselements. Of the accompanying figures:

FIG. 1 is a block diagram of a storage system according to an exampleembodiment;

FIG. 2 is a block diagram of a storage system according to anotherexample embodiment;

FIG. 3 is a block diagram of a storage allocator according to an exampleembodiment;

FIG. 4 is a method for managing files according to an exampleembodiment;

FIG. 5 is a method for managing the storage of discardable files in astorage device according to an example embodiment;

FIG. 6 is a method for marking one or more unsolicited files in aFAT32-structured file system according to an example embodiment;

FIG. 7 is an exemplary directory area associated with a FAT32 table;

FIG. 8 is a FAT32 table according to an example embodiment;

FIG. 9 is an NTFS table according to an example embodiment;

FIG. 10 is a logical image of a FAT-based file system according toexample embodiments;

FIG. 11 demonstrates files' storage management method in accordance withthe present disclosure;

FIG. 12 a illustrates an exemplary primary FAT;

FIG. 12 b illustrates an exemplary discardable FAT:

FIG. 13 is a flow chart of a method for managing a storage device usinga primary FAT and a discardable FAT;

FIG. 14 is a flow chart of a method for managing a storage device usinga FAT and a database;

FIG. 15 is a flow chart of a method for managing a storage device usinga FAT and a location file;

FIG. 16 illustrates an exemplary FAT including a cluster chain in whichan order of two or more clusters that comprise the cluster chain havebeen scrambled;

FIG. 17 illustrates an exemplary FAT and associated location files,where the FAT includes cluster chains in which an order of two or moreof the clusters that comprise the cluster chains have been scrambled;

FIG. 18 is a flow chart of a method for managing a storage device usinga FAT in which an order of two or more clusters that comprise a clusterchain is scrambled;

FIG. 19 is a flow chart of a method for utilizing conversion locks toprevent a conversion of a discardable file when the discardable file isopen in a file system implementing a primary FAT and, a discardable FAT;and

FIG. 20 illustrates exemplary bit masks user IDs in a file system.

DETAILED DESCRIPTION

The description that follows provides various details of exemplaryembodiments. However, this description is not intended to limit thescope of the claims but instead to explain various principles of theinvention and the manner of practicing it.

In order to address unsolicited content and related issues, user filesare given storage priority over other files, and a storage usage safetymargin is maintained to guarantee that priority. A “user file” is a filethat a user of a storage device has willingly stored, or has approvedits storage in the storage device. For example, a music file that theuser downloads to her/his storage device is regarded as a user file.Being requested or approved for storage by the user, user files areregarded as “solicited” files.

The “other files” are referred to herein as “publisher files” and“unsolicited files”. A “publisher file” is a file stored in a storagedevice without the user requesting it or being aware of it; at least notfor a while. The user may not want to use an unsolicited file. Unusedunsolicited files tend to consume expensive storage space on the user'sstorage device. Therefore, according to the principles disclosed hereinsuch files are permitted to be stored in the storage device only ifstoring them does not narrow the storage usage safety margin. Storagepriority is rendered to user files by maintaining a free storage space(i.e., a storage usage safety margin) that will be reserved for futureuser's files. The storage usage safety margin has to be maintained inorder to ensure that user files can be stored in the storage devicewhenever required or desired.

If for some reason the storage usage safety margin gets narrower thandesired, one or more unsolicited files will be removed (i.e., deleted)from the storage device in order to restore the storage usage safetymargin. Maintaining the storage usage safety margin guarantees storagespace for additional user files if such files are downloaded to thestorage device. To this end, unsolicited files are marked as“discardable” in a structure of the storage file system and, ifrequired, removed later to reclaim at least the free storage spacerequired to maintain the storage usage safety margin.

Because the likelihood of the user using the various discardable filesmay differ from one discardable file to another, each unsolicited file(i.e., each discardable file) is assigned in advance a discardingpriority level according to one or more criteria such as the probabilityof using the file, the probable revenue associated with using the file,the file's size, the file's type, the file's location, the file's age,etc. For example, the discarding priority level may be determined by thepotential for revenue. According to another example movie trailers oradvertisements would have a higher discarding priority than the actualmovie because users usually don't like seeing trailers andadvertisements. According to another example, the one or morediscardable files that are most likely to be used by the user will beassigned the lowest discarding priority level, which means that suchfiles will be the last file(s) to be removed from the storage device. Inother words, the higher the usage probability is of a discardable filethe lower the level is of the discarding priority level assigned to thatfile. If the desired storage usage safety margin is not fully restoredeven though one or more discardable files were removed, additionaldiscardable files will be removed from the storage device until thedesired storage usage safety margin is restored.

Briefly, a file system implements a methodology for storing andorganizing computer files. A file system includes a set of abstract datatypes and metadata that are implemented for the storage, hierarchicalorganization, manipulation, navigation, access, and retrieval of data.The abstract data types and metadata form “directory trees” throughwhich the computer files (also referred to herein as “data files”, or“files” for simplicity) can be accessed, manipulated and launched. A“directory tree” typically includes a root directory and optionalsubdirectories. A directory tree is stored in the file system as one ormore “directory files”. The set of metadata and directory files includedin a file system is called herein a “file system structure”. A filesystem, therefore, includes data files and a file system structure thatfacilitate accessing, manipulating, updating, deleting, and launchingthe data files.

File Allocation Table (“FAT”) is an exemplary file system architecture.FAT file system is used with various operating systems including DR-DOS,OpenDOS, MS-DOS, Linux, Windows, etc. A FAT-structured file system usesa table that centralizes the information about which storage areas arefree or allocated, and where each file is stored on the storage device.To limit the size of the table, storage space is allocated to files ingroups of contiguous sectors called “clusters”. As storage devices haveevolved, the maximum number of clusters has increased and the number ofbits that are used to identify a cluster has grown. The version of theFAT format is derived from the number of the table bits: FAT12 uses 12bits; FAT 16 uses 16 bits, and FAT32 uses 32 bits.

Another file system architecture is known as New Technology File System(“NTFS”). Currently, NTFS is the standard file system of Windows NT,including its later versions Windows 2000, Windows XP, Windows Server2003, Windows Server 2008, and Windows Vista. FAT32 and NTFS areexemplary file systems with which storage device 100 can be provided.

FIG. 1 shows a typical storage device 100. Storage device 100 includes astorage area 110 for storing various types of files (e.g., music files,video files, etc.), some of which may be user files and others may bepublisher files. Storage device 100 also includes a storage controller120 that manages storage area 110 via data and control lines 130.Storage controller 120 also communicates with a host device 140 via hostinterface 150. Host device 140 may be dedicated hardware or generalpurpose computing platform.

Storage area 110 may be, for example, of a NAND flash variety. Storagecontroller 120 controls all of the data transfers to/from storage area110 and data transfers to/from host device 140 by controlling, forexample, “read”, “write” and “erase” operations, wear leveling, and soon, and by controlling communication with host 140. Storage area 110 maycontain, for example, user files and publisher's files, protected datathat is allowed to be used only by authorized host devices, and securitydata that is used only internally, by storage controller 120. Hosts(e.g., host 140) cannot directly access storage area 110. That is, if,for example, host 140 asks for, or needs, data from storage device 100,host 140 has to request it from storage controller 120. In order tofacilitate easy access to data files that are stored in storage device100, storage device 100 is provided with a file system 160.

Storage area 110 is functionally divided into three parts: user area170, publisher area 180, and free storage space 190. User area 170 is astorage space within storage area 110 where user files are stored.Publisher area 180 is a storage space within storage area 110 wherepublisher files are stored. Free storage space 190 is an empty storagespace within storage area 110. Free storage space 190 can be used tohold a user file or a publisher file. Upon storing a user file in freestorage space 190, the storage space holding the user file is subtractedfrom free storage space 190 and added to user area 170. Likewise, uponstoring a publisher file in free storage space 190, the storage spaceholding the publisher file is subtracted from free storage space 190 andadded to publisher area 180. If a user file or a publisher file isremoved (i.e., deleted) from storage area 110, the freed storage spaceis added (it returns) to free storage space 190.

If the size of free storage space 190 permits it, the user of storagedevice 100 can download a user file from host 140 to storage area 110.The downloaded user file will be stored in free storage space 190 and,as explained above, the storage space holding that file will besubtracted from free storage space 190 and added to user area 170. Asexplained above, user files have priority over other (e.g., publisher)files, and in order to guarantee that priority, a desired storage usagesafety margin is set, and, if required, restored, in the way describedbelow.

Host 140 includes a storage allocator 144 to facilitate restoration offree storage space 190. Storage allocator 144 may be hardware, firmware,software or any combination thereof. In general, storage allocator 144determines whether a file (e.g., file 142) that is communicated to host140 is either a user file or a publisher file, and then marks thecommunicated file accordingly (i.e., as a non-discardable file or as adiscardable file).

If storage allocator 144 determines that a file (e.g., file 142)communicated to host 140 is non-discardable, for example because thefile is a user file, storage allocator 144 stores the file in storagearea 110 in a regular way. As explained above, the storage space withinstorage area 110 that holds the non-discardable file will be added to,or be part of, user area 170. If, however, storage allocator 144determines that the file communicated to host 140 is discardable, forexample because it is a publisher file, storage allocator 144 marks thefile as discardable. If free storage space 190 is larger than thedesired storage usage safety margin storage allocator 144 also storesthe marked discardable file in free storage space 190, and, as explainedabove, the storage space within free storage space 190 that holds thediscardable file is subtracted from free storage space 190 (i.e., thefree storage space is reduced) and added to publisher area 180 (theaddition is logically shown as discardable file(s) 182).

As explained above, the likelihood that publisher files may be used bythe user may vary from one publisher file to another, which makes apublisher file with the least usage likelihood the first candidate forremoval form storage area 110. Therefore, in addition to marking a fileas non-discardable or discardable storage allocator 144 assigns adiscarding priority level to each discardable file prior, concurrently,or after the discardable file is stored in storage area 110.

By marking files as non-discardable or as discardable, assigning adiscarding priority level by storage allocator 144, and by using thefile system 160 (or an image thereof) of storage device 100, storageallocator 144 “knows” the number of user files and publisher files instorage area 110, and also their sizes and logical locations withinstorage area 110. Knowing this information (i.e., the number, sizes andlocations of the files), and particularly based on one or more markedfiles, storage allocator 144 manages storage area 110 and the storage ofsolicited and unsolicited files in storage area 110. Managing storagearea 110, or managing storage of files in storage area 110, may include,for example, restoring a storage usage safety margin by selectivelyremoving one or more files marked as discardable, freeing a storage areaby removing all files marked as discardable, and remapping clusters of afile to a lower-performance storage module. Managing storage area 110 orfiles stored therein may include managing other, additional, oralternative aspects of storage area 110 or files stored therein.

Storage allocator 144 also knows, by the discarding level assigned toeach discardable file, the order at which discardable files can orshould be discarded (i.e., deleted or removed from storage area 110) inorder to restore the free storage space originally reserved for futureuser files (i.e., to restore the desired storage usage safety margin).Accordingly, if a user wants to store a new user file in storage area110 but there is not enough free storage space to accommodate that userfile (which means that the storage usage safety margin is narrow thandesired), storage allocator 144 uses the discarding priority levelsassigned to the discardable files to iteratively delete one discardablefile after another to regain more free storage space (i.e., to extendfree storage space 190) until the desired storage usage safety margin isfully restored. As explained above, a fully restored storage usagesafety margin guarantees with high probability that an adequate freestorage space is reserved for future user files. Discardable files areremoved or deleted from storage device 100 only responsive to receivinga request to store a new user files because it is taken into accountthat the user may want to use a stored discardable file sometime and,therefore, the discardable file is removed from the storage device onlyif the storage space accommodating that file is required for the newuser file. Storage allocator 144 may be embedded or incorporated intohost 140, or it may reside externally to host 140 (shown as dashed box144′) and to storage device 100.

Storage allocator 144 has a representative image of the file system of,or associated with, storage device 100. Storage allocator 144 uses thestorage device's file system image to mark files as non-discardable oras discardable, and to assign a discarding level to each discardablefile. In one example, the file system includes the FAT and in this casethe marking is done in an unused portion of a FAT entry associated withthe file, by setting one or more unused bits. Because different filesystems have different structures, marking files (i.e., asnon-discardable or as discardable) and assigning discarding levels isadapted to the used file system structure, as elaborated in anddescribed below in connection with FIGS. 6 through 1.

FIG. 2 is a block diagram of a portable storage device 200 according toanother example embodiment. Storage controller 220 functions likestorage controller 120 and storage allocator 244 functions like storageallocator 144. Storage allocator 244 may be hardware, firmware, softwareor any combination thereof. Storage allocator 244 internally cooperateswith storage controller 220. Whenever storage controller 220 receivesfrom host 240 a storage request to store a file in storage area 210, therequest including an indication of whether or not the file is adiscardable file, storage controller 220 informs storage allocator 244of the storage request and whether or not the file is discardable. Thestorage allocator 244 then marks the file either as non-discardable ordiscardable in the structure of the file system associated with storagedevice 200. Typically, applications running on the host 240 determinethat a file is a discardable file and send a flag or other indication tothe storage controller 220 indicating that the file is a discardablefile. The applications running on the host 240 send the flag or otherindication as part of storage protocols for requesting to store a fileon the storage device. Examples of such storage protocols include POSIXfile system functions or the usage of the java.io class tree.

If storage allocator 244 determines that the new file is discardablestorage allocator 244 assigns to the new file a discarding prioritylevel according to the file's usage probability. Then, storage allocator244 evaluates the current size of free storage space 290 and decideswhether one or more discardable files should be removed (i.e., deleted)from storage area 210 in order to make room for the new file. Ifdiscardable file or files should be removed from the storage devicestorage allocator 244 decides which file(s) are the current candidatefiles for removal. Then, storage allocator 244 notifies storagecontroller 220 of the discardable files that should be removed fromstorage area 210 and, responsive to the notification, storage controller220 removes the discardable file or files indicated by storage allocator244. In some configurations of portable storage device 200, the storageallocator 244 may be functionally disposed between storage controller220 and storage area 210. In configurations where storage allocator 244is functionally disposed between storage controller 220 and storage area210, storage allocator 244 or storage area 210 have to assume some ofthe functions of storage controller 220. In such configurations storagearea 210 is comprised of memory units that communicate at a higher levelthan flash NAND protocols.

FIG. 3 is a block diagram of a storage allocator 300 according to anexample embodiment. Storage allocator 300 includes a memory unit 310, aprocessor 320, and an interface 330. Memory unit 310 may hold a filesystem structure, or an image of the file system structure, that isassociated with a storage device (e.g., storage device 200 of FIG. 2).Processor 320 manages the file system associated with the storagedevice. Interface 330 may be adapted to cooperate with a host and with astorage controller of a storage device, as demonstrated in FIG. 1, oronly with a storage controller of a storage device, as demonstrated inFIG. 2.

Processor 320 is configured or adapted to receive a request viainterface 330 to store a file in a storage area of the storage device,and to mark the file either as discardable or as non-discardable in astructure of the file system associated with the storage device withwhich storage allocator 300 operates. If interface 330 is functionallyattached to storage controller 220 of FIG. 2 (and thus receives, forexample, SCSI or wrapped USB/MSC commands rather than file levelcommands), the received request is at a level that is much lower than afile level. That is, the received request would be a request to storesectors at logical block addresses that, when properly interpreted by ahost, would correspond to a file. If storage controller 220 supports theNVMHCI protocol, or a networking file system protocol such as NFS or asimilar protocol, storage controller 220 can get file level requests.Therefore, the communication between a storage controller such asstorage controller 220 and an interface such as interface 330 is notlimited to NVMHCI or to NVMHCI-like implementations. Communicationinterface 330 may be an integral of storage allocator 300, as shown inFIG. 3.

Processor 320 is further configured or adapted to send the marked fileto the storage device, marking the file as discardable includesassigning to the file a discarding priority level. If the file systemused by the storage device is FAT-based, processor 320 assigns thediscarding priority level to the marked file by setting a correspondingvalue to m uppermost (i.e., most significant) bits (e.g., m=4) in a FATcorresponding to the marked file. The corresponding value set to themost significant bits in the FAT entry, or the value set to the NTFSdirectory entry, may be, or it may be, related to an attribute of thefile. By “attribute” is meant a metadata tag or some data structure inthe header of the FAT table or NTFS table that contains information thatpertains to the type of the content stored within the table.“Advertisement”, “premium content”, and “promotional (free) content” areexemplary types of contents that may be stored in the FAT table or inthe NTFS table. Alternative criteria for setting discarding levels are,for example, the last accessed files, file sizes, file types, etc.

The number m of the uppermost bits of FAT32 entries dedicated formarking files may be four or less than four because those bits are notused. In addition, the more bits are used the more discarding prioritylevels can be used. For example, using three bits (i.e., m=3) provideseight (2³=8) discarding priority levels and using four bits (i.e., m=4)provides sixteen (2⁴=16) discarding priority levels (i.e., includingdiscarding priority level “0”, which is assigned to non-discardablefiles). In other words, processor 320 sets the value of the m uppermostbits to 0 if the marked file is non-discardable or to a value between 1and 2^(m)−1 if the marked file is discardable. The discarding prioritylevel indicates the priority at which the marked file can or should bediscarded from the storage device. For example, depending on theimplementation, the value “1” may denote a file that is eitherdiscardable with the lowest priority or with the highest priority, andthe value “2^(m)−1” may respectively denote a file that is eitherdiscardable with the highest priority or with the lowest priority.

Processor 320 may assign the discarding priority levels to marked filesaccording to an anticipated usage of the files, as explained above inconnection with the likelihood or probability that an unsolicited fileis going to be used by the user of the storage device. Processor 320 mayupdate the discarding priority level of the marked file with, orresponsive to receiving, each request to store a new file in the storagedevice. Processor 320 may update the discarding priority level of agiven marked file independently from one or more new requests to store afile in the storage device. For example, a file that was previously of ahigh priority may have its priority lowered after a certain timeinterval. Processor 320 deletes a file that is stored in the storagedevice if the file has associated with it a discarding priority levelthat equals or is greater than a predetermined discarding thresholdvalue. Processor 320 may (re)set the discarding threshold value based onthe number of file writes or additions, or depending on the anticipateduse of free storage space on the storage device or availability of newpublisher files.

Memory unit 310 may hold an assignment table 340 that containsdiscarding priority levels that processor 320 assigns to files stored inthe storage device. In addition, assignment table 340 may hold files'identifiers and information that associates files with the discardingpriority levels assigned to the files. Assignment table 340 mayadditionally hold a discarding threshold value. The information held inassignment table 340 allows processor 320 to identify which discardablefile or files can be removed form the storage device in order to restorethe desired storage usage safety margin.

Responsive to receiving a request to store a new file in the storagedevice processor 320 evaluates the size of a free storage space (f) onthe storage device and stores the new file in the storage device if theevaluated size of the free storage space on the storage device is largerthan a predetermined size or, if it is not larger than the predeterminedsize, processor 320 searches for one or more discardable files withinthe storage device that can be deleted and, upon finding such file orfiles, processor 320 deletes that file or files to extend the currentfree storage space (f) such that the total size of the extended freestorage space equals or is larger than the predetermined size. Thediscardable file or files can be deleted from the storage device if thediscarding priority level associated with the discardable files equalsor is greater than a predetermined discarding threshold value (forexample between 1 and 15 inclusive, for example 15).

After the free storage space is extended enough processor 320 permitsthe new file to be stored in the extended free storage space. By “freestorage space is extended enough” is meant expanding the free storagespace by freeing one occupied storage space after another until thetotal free storage space can accommodate the new file without narrowingthe desired storage usage safety margin mentioned above or,equivalently, until the total size of the extended free storage spaceequals or is greater then a predetermined size or until all discardablefiles are removed.

Processor 320 can be a standard off-the-shelf System-on-Chip (“SoC”)device or a System-in-Package (“SiP”) device or general purposeprocessing unit with specialized software that, when executed, performsthe steps, operations and evaluations described herein. Alternatively,processor 320 can be an Application-Specific Integrated Circuit (“ASIC”)that implements the steps, operations and evaluations described hereinby using hardware.

FIG. 4 is a method for storing discardable files according to oneexample embodiment. FIG. 4 will be described in association with FIG. 1.At step 410 host 140 receives a request to store file 142 in storagedevice 100. At step 420 storage allocator 144 marks the file as“discardable” or as “non-discardable” and sends, at step 430, the markedfile to storage controller 120 of storage device 100 (i.e., for storagein storage area 110) if free storage space 190 is sufficiently large. Afile is marked also in the sense that a discarding priority level isassigned to the file. At step 440 storage allocator 144 manages storagearea 110 (through communication with storage controller 120) or filesthat are stored in storage area 110 based on the marked file and,optionally, based on one or more files that have already been marked.

FIG. 5 is a method for managing the storage of discardable files in astorage device according to one example embodiment. FIG. 5 will bedescribed in association with FIG. 1. A new file is candidate forstorage in storage device 100. Knowing the current image of file system160 of storage device 100, storage allocator 144 evaluates, at step 510,the current size “f” of free storage space 190 to see whether freestorage space 190, whose current size is f, can accommodate the new file(i.e., the file that is candidate for storage). In general, the waystorage allocator 144 handles the new file depends on whether the newfile is a user file or a publisher file. Therefore, storage allocator144 determines first whether the new file is a user file or a publisherfile.

The New File is a User File

At step 520 storage allocator 144 checks whether free storage space 190can accommodate the new user file. If free storage space 190 canaccommodate the new user file (shown as “Y” at step 520), storageallocator 144 stores, at step 560, the new user file in free storagespace 190 regardless of whether the desired storage usage safety marginis narrowed by storing the new user file or not. If the desired storageusage safety margin gets narrower (i.e., relative to the desired storageusage safety margin) after storage allocator 144 stores the new userfile in free storage space 190, storage allocator 144 takes no furtheractions with respect to the storage of the new user file.

If, however, the desired storage usage safety margin gets narrower afterstorage allocator 144 stores the new user file in free storage space190, step 550 includes an additional step where storage allocator 144determines which stored discardable file should be deleted first, whichdiscardable file should be deleted second, and so on, in order tomaintain the desired storage usage safety margin. Storage allocator 144determines which discardable file should be deleted first, which shouldbe deleted second, etc. based on discarding levels that storageallocator 144 assigned to the stored discardable files.

If storage allocator 144 determines at step 520 that free storage space190 cannot accommodate the new user file (shown as “N” at step 520),storage allocator 144 determines, at step 530, whether free storagespace 190 and the storage space consumed by discardable files, whencombined, is sufficient for storing the new user file. If the combinedstorage space is insufficient (shown as “N” at step 530), this meansthat no matter how many discardable will be deleted the new user filecannot be stored in the “non-user” storage area due to its larger size.If the combined storage space is sufficient (shown as “Y” at step 530),storage allocator 144 searches, at step 540, among stored discardablefiles which discardable file can be deleted in order to free sufficientstorage space for the new user file. Storage allocator 144 searches forthese discardable files by using the file system of storage device 100because, as explained above, storage allocator 144 marks files asnon-discardable or as discardable in the file system of the storagedevice. In addition, the discarding levels assigned by storage allocator144 to marked files are also embedded into the storage device's filesystem such that each discarding level is associated with thecorresponding marked file.

Upon finding a discardable file (“DF”) that should be discarded first(that file is called hereinafter “DF1”), storage allocator 144 deletesfile DF1 in order to add, or to return, its storage space (that storagespace is called hereinafter “SP1”) to storage space 190.

Then, at step 550 storage allocator 144 checks whether the extended freestorage space 190 (i.e., free storage space 190 plus the last returnedstorage space, or f+SP1) can accommodate the new user file. If theextended free storage space 190 (i.e., f+SP1) still cannot accommodatethe new user file (shown as “N” at step 550) storage allocator 144iteratively repeats step 550 (the iterations are shown at 555) in orderto return an additional storage space to free storage space 190 (i.e.,by finding and deleting the next discardable file that should bedeleted).

Upon finding the next discardable file with the second highestdiscarding priority (the next discardable file is called hereinafter“DF2”), storage allocator 144 deletes file DF2 in order to free and addadditional storage space (the additional storage space is calledhereinafter “SP2”) to free storage space 190. Then, at step 550 storageallocator 144 checks again whether the extended free storage space 190(i.e., free storage space 190 plus the two last freed storage spaces, orf+SP1+SP2) can accommodate the new file. If the extended free storagespace 190 (i.e., f+SP1+SP2) still cannot accommodate the new file (shownas “N” at step 540), storage allocator 144 repeats step 540 one moretime in order to find the next discardable file that should be deleted.Storage allocator 144 iterates steps 540 and 550 until the accumulatedfree storage space 190 can accommodate the new user file (shown as “Y”at step 550). Then, at step 560 storage allocator 144 stores the newuser file in storage area 110.

As said above, if the actual storage usage safety margin gets narrowerthan the desired storage usage safety margin after storage allocator 144stores the new user file in free storage space 190, step 560 may includean additional step in which storage allocator 144 determines whichstored discardable file should be deleted first, which discardable fileshould be deleted second, etc., in order to restore the desired storageusage safety margin.

The New File is a Publisher File

If the new file is a publisher file, storage allocator 144 stores (atstep 560) the new publisher file in storage area 110 only if freestorage space 190 can accommodate the new publisher file withoutnarrowing the desired storage usage safety margin. That is, if storingthe new publisher file would result in narrowing the desired storageusage safety margin storage allocator 144 may decide not to store thenew publisher file in storage area 110. In such a case, storageallocator 144 may refrain from taking any action with respect to thatfile, and delete no file from the storage device to free storage spacefor the new publisher file. Alternatively, storage allocator 144 maydelete at step 540 one or more higher priority discardable files inorder to free storage space for a discardable file that has a lowerdiscarding priority. As stated above, files are marked in, anddiscarding levels are embedded into, the file system of storage device100, and the way the files are marked and the discarding levels embeddedinto the file system depends on, or can be adapted to, the used filesystem.

FIG. 6 is a method for marking an unsolicited file in a FAT32-structuredfile system according to one example embodiment. FAT32-structured filesystems use clusters. As described above in connection withFAT32-structured file systems, the number of bits that are used toidentify a FAT32 cluster is 32. FIG. 6 will be described in associationwith FIG. 1.

At step 610 m uppermost bits of the 32 bits (where m≦4) of each clusterof the FAT32 are allocated or dedicated for marking files asnon-discardable or as discardable, as the case may be, and also forholding a corresponding discarding level for each discardable file.Assigning the discarding level to a file is done by setting acorresponding value to the allocated m bits corresponding to the markedfile.

At step 620 storage allocator 144 evaluates the level of likelihood atwhich the user of storage device 100 will use the unsolicited file.Evaluation of the likelihood of using the file can be implemented invarious ways that are known to those skilled in the art of consignmentfiles. For example, the evaluation of the likelihood of using the filemay be based on monitoring the location of the person using the storagedevice, and/or on monitored user's previous experience and preferences.Evaluation of the likelihood of using the file may also be based, forexample, on the type of content stored within the FAT table or NTFStable (e.g., “advertisement content”, “premium content”, “promotional(free) content”, etc.). Storage allocator 144 may use alternative oradditional criteria to evaluate the likelihood at which the file will beused. For example it may use attributes or characteristics of file(s),which may be, or be associated with, the last accessed file(s), filesizes, file types, etc.

After storage allocator 144 evaluates the level of likelihood at whichthe user will use the unsolicited file storage allocator 144 assigns, atstep 630, a discarding priority level corresponding to the evaluatedlikelihood level of usage of the unsolicited file. The more likely theunsolicited file is going to be used by the user of storage device 100the lower is the discarding level.

If m equals four bits, this means that the discarding scale provides 15discarding levels from 1 (i.e., 0001) to 15 (i.e., 1111). That is,discarding level 0 will be assigned to every non-discardable file,discarding level 1 will be assigned to a discardable file with thelowest discarding priority, and discarding level 15 will be assigned toa discardable file with the highest discarding priority. After storageallocator 144 assigns a corresponding discarding level to theunsolicited file, storage allocator 144 sets, at step 640, acorresponding value between 1 and 15 to the four uppermost bits of theclusters associated with the unsolicited file. If the unsolicited filehas associated it two or more clusters, the four uppermost bits in eachcluster is set to the same value.

At step 650 it is checked whether the unsolicited file is the last filethat needs to be evaluated. If the unsolicited file is not the last filethat needs to be evaluated (shown as “N” at step 650) another file isevaluated in the way described above. If the unsolicited file is thelast file that needs to be evaluated (shown as “Y” at step 650) theunsolicited file(s) is(are) sent to storage device with the m bits foreach whose value was set at step 640.

FIG. 7 is an exemplary directory table 700 associated with a FAT32table. Directory table 700 is only a partial table used for illustrationand as such, table 700 does not show all the fields of a FAT directoryentry. Directory area 700 holds particulars of files that are stored ina related file system, such as the files names, files size, and where ina related storage space each file begins. The particulars of the filesare held in the following fields. Field 710 holds the Disk OperatingSystem (“DOS”) filenames of the files stored in the related file system,field 720 holds the extension of the files, field 730 holds variousattributed of the files, field 740 holds the high 16-bitword of theFirst Cluster Number (“FCN”) of the files, field 750 holds the low partof the First Cluster Number (“FCN”) of the files, and field 760 holdsthe size of the files. Each FCN number indicates the first logicalcluster where a file may be found.

The first entry of directory area 700 holds information for an exemplaryfile called “REALFILE” (shown at 770). REALFILE 770 has a file extension“DAT”, its FCN is “0000 0002” (shown at 755), and its size is “000024E4”. Numbers in table 700 are shown in hexadecimal values. As part ofthe standard, attribute values “00” (shown at 780) and “20” (not shownin FIG. 7) refer to a “regular” file, whereas attribute value “02”refers to a file that is hidden in the file system. Filename“\xE5Consign” indicates a deleted file, where “\xE5” means that thevalue of the first byte of the filename is E5 in hex. By way of example,FCN number 0000 0002 (shown at 755) designates the first cluster of fileREALFILE.

FIG. 8 is an exemplary partial FAT32 table 800 according to an exampleembodiment. FAT32 table 800 is shown as a double-word (“DWORD”) array,and the values are hexadecimal values. Reference numeral 810 designatesthe type of device holding FAT32 table 800, where “F8” refers to a harddrive. FAT32 table 800 includes 23 clusters that are designated ascluster #1 (shown at 820), cluster #2 (shown at 825), . . . , andcluster #23 (shown at 830). FIG. 8 will be described in association withFIG. 7. A cluster in FAT32 table 800 may be the first cluster of a file,or it may point to the next linked cluster of the file, or it may be anEnd-of-File (“EOF”) indication.

Referring again to directory area 700, the first FCN of file REALFILE(shown at 770) is “0000 0002” (shown at 755), which points at cluster #2in table 800 of FIG. 8. As shown in FIG. 8 the value of cluster #2(i.e., the value “000 0003”) points (shown at 840) at cluster #3, whichis the next file's cluster. Likewise, the value of cluster #3 (i.e.,“0000 0004”) points at cluster #4, which is the next file's cluster.Cluster #4 has the value “OFFF FFFF” (“F” is the hexadecimal digit thatrepresents the decimal value “15”), where “FFF FFFF” (shown at 850)denotes the file's EOF indication, and the zero value (shown at 860)denotes discarding level 0. File REALFILE, therefore, has associatedwith it three clusters (i.e., cluster #2, cluster #3, and cluster #4).

As explained above, a discarding level 0 is assigned to non-discardablefiles. It is noted that the most significant hexadecimal digit of eachcluster of a particular file is set to the same discarding prioritylevel that is assigned to that file. For example, file REALFILE has beenassigned a discarding level “0” and, therefore, each of the mostsignificant hexadecimal digits of clusters #2, #3, and #4 has that value(i.e., value “0”, the “0” values are underlined). According to anotherexample, the file “E5 Consign” whose FCN is “0000 0005” (as shown inFIG. 7) has been assigned a discarding priority level “1”. Therefore,the most significant hexadecimal digit of each of clusters #5 through12, which pertain to that file, has the value “1” (for example as shownat 870). In other words, according to the present disclosure the mostsignificant hexadecimal digit (or, equivalently, the four uppermost bitsof the clusters associated with a particular discardable file are set tothe same value corresponding to the discarding priority level assignedto the particular file. As explained above the number m of the uppermostbits used for indicating the discarding priority level may differ fromfour (i.e., m≦4).

FIG. 9 is an exemplary partial NTFS table 900 according to an exampleembodiment. NTFS table 900 holds particulars of files such as the filenames, the file sizes, etc. NTFS table 900 includes a data field 910 tohold “regular” data (e.g., data 920) for files that change according to“normal” data flow. According to the present disclosure, NTFS table 900also includes a “Discarding Information” field 915 for holding,discarding information (e.g., discarding information 930) for eachevaluated file. Discarding information field 915 may also includeinformation other than the discarding priority level. For example,discarding information field 915 may include information pertaining tothe server that supplied the file and an expiration time after which thefile must be discarded. Unlike FAT-based file systems, in NTFS-basedfile systems the discarding values assigned to discardable files are notlimited to a maximum number that is dictated by a set of bits. Thismeans that the range of discarding values can be chosen liberally. Forexample, discarding values can range from 1 to 25. NTFS is an exemplarynon-FAT file system. In general, corresponding discarding values may beset to a data field in a non-FAT based file system entries correspondingto marked files.

FIG. 10 is a logical arrangement of file system 1000 of a storage deviceaccording to an example embodiment. A storage allocator (e.g., storageallocator 144 of FIG. 1) may either hold file system 1000 of the storagedevice with which it operates or an image of file system 1000, or thestorage allocator may have an access to file system 1000.

File system 1000 includes a boot section 1010, a FAT 1020 associatedwith file system 1000, directory tables 1030, a files area 1040, and adiscardable files area 1050. FAT 1020 includes a discardable filesallocations area 1025 that contains the discarding priority levels ofdiscardable files. Directory tables 1030 include access information foraccessing whatever files (i.e., discardable files and/or non-discardablefiles) are stored in the storage device. Files area 1040 contains thenon-discardable files. Index and database area 1045 holds indexes forthe discardable files and also metadata that is related to thediscardable files. The indexes and metadata held in Index and databasearea 1045 are used to calculate the discarding levels but they are notrequired during the actual discarding process. Discardable files area1050 holds the discardable files.

FIG. 11 demonstrates the files management method according to thepresent disclosure. FIG. 11 will be described in association withFIG. 1. It is assumed that at time T0 two user files (i.e., files “F1”and “F2”) are initially stored in storage area 110. Because files “F1”and “F2” are user files they are stored in user area 170 and thediscarding level assigned to them by storage allocator 144 is zero.Because the total storage capacity of storage area 110 is T (shown at1110) and files F1 and F2 are stored in storage device 100, the size ofthe remaining free storage space 190 (see FIG. 1) is f (shown at 1120).It is assumed that a publisher wants to store three unsolicited files instorage area 110. As described above, storage allocator 144 evaluatesthe size of free storage space 190 (or f at 1120) in storage device 100in order to determine whether storing the publisher's three unsolicitedfiles in storage area 110 will not narrow a desired storage usage safetymargin (shown at 1130) that is reserved for future user's files. Ifstoring publisher's three unsolicited files would narrow storage usagesafety margin 1130 (i.e., the desired storage usage safety margin)storage allocator 144 will refrain from storing these files.

In this example, storage allocator 144 determines that the publisher'sthree unsolicited files can be stored in storage area 110 withoutreducing storage usage safety margin 1130. Therefore, at time T1 storageallocator 144 permits storage controller 120 to store the publisher'sthree unsolicited files in storage area 110. The three publisher'sunsolicited files are designated as “P1”, “P2”, and “P3”. Storageallocator 144 also determines the probability that files P1, P2, and P3will be used by the user of storage device 100 and assigns acorresponding discarding level to each of these file. Storage allocator144 then stores the discarding levels assigned to the files in the FATtable, as demonstrated in FIG. 8, or in the NTFS table, as demonstratedin FIG. 9.

At time T2 the user of storage device 100 wants to store in storage area110 two more files (i.e., files “F3” and “F4”). Storage allocator 144reevaluates the size of free storage space 190 (or f at 1120) in storagedevice 100 in order to determine whether there is sufficient storagespace in storage area 110 to store the additional files (i.e., files F3and F4). In this example storage allocator 144 determines that thecurrently free storage space can accommodate files F3 and F4. Therefore,at time T2 storage allocator 144 permits storage controller 120 to storefiles F3 and F4 in storage area 110.

Because files F3 and F4 are user files the probability that files F3 andF4 will be used by the user of storage device 100 is irrelevant becauseuser files have storage priority over publisher files regardless of howmany times, if at all, the user is going to use files F3 and F4.Accordingly, storage allocator 144 assigns a discarding level “0” tofiles F3 and F4 and stores the assigned discarding level in the FATtable, as demonstrated in FIG. 8, or in the NTFS table, as demonstratedin FIG. 9.

At time T3 the user of storage device 100 wants to store in storage area110 another file (i.e., file “F5”). Storage allocator 144 reevaluatesthe size of free storage space 190 (or fat 1120) in storage device 100in order to determine whether there is sufficient storage space instorage area 110 to store the additional file (i.e., file F5).

In this example, storage allocator 144 determines that the currentlyfree storage space can accommodate file F5. Therefore, at time T3storage allocator 144 permits storage controller 120 to store file F5 instorage area 110. As shown in FIG. 11, storing user file F5 narrows thestorage usage safety margin. That is, the free storage space fin storagearea 110 that remains after files F1 through F5 and P1 through P3 arestored in storage area 110 is smaller than storage usage safety margin1130. Therefore, storage allocator 144 reinstates or restores thestorage usage safety margin by removing one of the publisher's files(i.e., P1, P2, and P3). A storage usage safety margin is reinstated orrestored by removing (i.e., deleting) one or more publisher filesbecause, as explained above, user files have ultimate storage priority.

As described above, the decision which publisher file or publisher filesshould be removed from the storage area 110 is made by storage allocator144 based on the discarding priority level that storage allocator 144assigned to each stored discardable file.

Turning back to FIG. 11, it is assumed that among the stored publisherfiles P1 through P3 publisher file P3 was assigned the highestdiscarding priority level (e.g., 13). Therefore, at time T4 file P3 isremoved from storage area 110, thus enlarging the free storage space190. Because the size of free storage space 190 (or fat 1120) at time T4is larger than storage usage safety margin 1130, there is no need toremove any more publisher files.

The user of storage device 100 may want to remove one or more userfiles. At time T5 the user removed two of his files (i.e., files F4 andF5), thus further enlarging free storage space 190. The removal of filesF4 and F5 has nothing to do with the size of free storage space 190 orthe storage usage safety margin because, as stated herein, regainingfree storage space or restoring the storage usage safety margin is doneby removing as many discardable files as necessary. It is assumed that apublisher wants to store another unsolicited file in storage area 110.As described above, storage allocator 144 evaluates the size of freestorage space 190 (or f at 1120) in order to determine whether storingthe publisher's unsolicited file in storage area 110 will not narrowstorage usage safety margin 1130. If storing the publisher's the newunsolicited file will narrow storage usage safety margin 1130 storageallocator 144 will refrain from storing that file.

In this example storage allocator 144 determines that the publisher'snew unsolicited file (i.e., file “P4”) can be stored in storage area 110without reducing storage usage safety margin 1130. Therefore, at time T6storage allocator 144 permits storage controller 120 to store thepublisher's file P4 in storage area 110. Storage allocator 144 alsodetermines the probability that file P4 will be used by the user ofstorage device 100 and assigns a corresponding discarding level to thisfile. Storage allocator 144 then stores the discarding level assigned tofile P4 in the FAT table, as demonstrated in FIG. 8, or in the NTFStable, as demonstrated in FIG. 9. The process of storing new publisher'sfiles and new user files and removing stored files may continue whileeach time a new file is to be added to storage area 110 storageallocator 144 evaluates the current size of free storage space 190 anddetermines which publisher file or files (if at all) has/have to beremoved from storage area 110.

Assigning a discarding level to a discardable file may be based on userexperience or preferences, on Global Positioning System (“GPS”) locationof the user, and/or on other criteria. For example, if the user of thestorage device seems (based on previous user experience) to like certaintypes of music, the storage allocator may assign a relatively lowdiscarding priority level (e.g., 3 in a scale of 1 to 15) to apublisher's file if that file contains music that is one of the user'sfavorite types of music. However, if the publisher's music is dislikedby the user (i.e., based on previous user experience), the storageallocator may assign to the related publisher's file a higher discardingpriority level (e.g., 12 in a scale of 1 to 15). The criteria used toassign a discarding level to a discardable file may include anticipatedusage of the file, anticipated revenue associated with using the file,the file's type, the file's size, the file's location in the storagedevice, the file's age, and other criteria or parameter as specifiedherein. Other criteria, whether alone or in combination with any of thecriteria mentioned herein, may likewise be used, and the assignment ofdiscarding levels may be done using one or more criterions. In addition,different criteria may be used to assign a discarding level to differentdiscardable files.

In another example, if a publisher wants to send to a user alocation-dependent advertisement (i.e., an advertisement relating to aproduct or service rendered within a specific locality), the storageallocator may assign a discarding priority level to the publisher'sadvertisement that changes according to the user's changing location.That is, the farther the user gets from a particular location, thehigher the discarding level would be, because by getting away from thespecific locality it can be assumed that the user is not interested inconsuming the product or service rendered at the specific locality.

As described above, cluster chains for discardable files are recorded ina FAT with a flag identifying a file associated with a FAT32 entry as adiscardable file. Typically, the flag is in the four most significantbits of each FAT32 entry. Because cluster chains may be allocated to adiscardable file but do not have a non-discardable file associated withthem, it is possible that a utility such as chkdsk or fsck.vfat willturn a discardable files into non-discardable files, also known as“real” files, thereby reducing the security of the file system 160.Additionally, there is a risk that some FAT recovery utilities willreset the discardable-file flags in the FAT32 entries. FAT32 file systemchecking and repair utilities often step through a file system and applyrules in order to fix common errors. Generally, these utilities may lookfor cluster chains in a FAT that have no corresponding entry in theFirst Cluster Number (FCN) column within the directory tables. Theutilities treat cluster allocations in the FAT that do not have anydirectory or file entries as unaccounted data fragments (also known asorphan clusters) and the utilities may delete these orphan clusters orcreate a corresponding file entry in a directory table. Because thediscardable file system described herein may make use of what wouldotherwise be considered an orphan cluster, the utilities may improperlyturn a discardable file into a non-discardable file or remove thediscardable file entirely.

To address these problems, in some implementations, the storageallocator 144 may associate a discardable file with a cluster chain in aprimary FAT, where the cluster chain hides a physical location of thediscardable file, and the storage allocator 144 stores the physicallocation of the file in a discardable FAT, a database, or one or morelocation files. Typically, the discardable FAT, database, or one or morelocation files are not visible to the primary FAT, and in someimplementations, an attribute associated with the discardable FAT,database, or one or more location files may be enabled that prevents ahost operating system from accessing the discardable FAT, database, orone or more location files.

As noted before, each entry in a FAT32 is 32 bits, but only the lower 28bits are used. Typically, the upper four bits are reserved and are setto zero. (Compliant implementations of FAT32 are required to ignore theupper four bits if set on allocated clusters, and to set the upper fourbits to zero when writing new FAT entries.) Discardable files aredistinguished from non-discardable files by a flag within the upper fourbits of the FAT entries of each cluster chain that is associated withthe file. Standard FAT32 drivers will see discardable files as allocatedspace and will not write over them. However, a storage allocator 144 mayperiodically perform operations, such as those described above withrespect to FIG. 5, in order to maintain free space allocations in thestorage device 110 and may recover the space allocated to discardablefiles.

By utilizing a primary FAT and at least one of a discardable FAT, adatabase, and one or more location files, the primary FAT may beextended. When the extended primary FAT is used in conjunction with abranch in file allocation table lookup logic, such that if the upperfour bits of a FAT entry are nonzero, information in the discardableFAT, database, or one or more location files reflecting a physicallocation of the discardable file is used in place of the FAT entry inthe primary FAT. Due to the information in the discardable FAT,database, or one or more location files overriding a value in the FATentry of the primary FAT, utilities such as chkdsk and fsck.vfat willnot turn discardable files into non-discardable files because theutilities will see the clusters of the discardable file as associatedwith directory or file entries in the discardable FAT, database or oneor more location files. Also, FAT recovery utilities will not reset theflags in FAT32 entries indicating that a file is a discardable filebecause utilities such as chkdsk and fsck.vfat see the clustersassociated with the discardable files as associated with directory orfile entries in the discardable FAT, database, or one or more locationfiles rather than as free space.

When the file system 160 utilizes a primary FAT 1200 and a discardableFAT 1201, to store a file that has been marked as a discardable file,the storage allocator 144 updates the primary FAT 1200 as shown in FIG.12 a to associate a cluster chain 1202 allocated to a discardable filewith the file. Generally, the cluster chain 1202 may be the same sizeas, or larger than, the discardable file associated with the clusterchain 1202. In some implementations, the cluster chain 1202 masks aphysical location of the discardable file in the primary FAT. Typically,as described above with respect to FIGS. 7 and 8, each cluster in thecluster chain starting in entry 1204 points to the next sequentialcluster of the cluster chain 1202 until a value such as 1FFF FFFF, asshown in entry 1206, indicates an end to the cluster chain 1202.However, in other implementations, each cluster of the cluster chain mayhave a value such as 1FFF FFFF indicating that the cluster is anindividually allocated cluster rather than pointing to a next sequentialcluster of a cluster chain.

The first entry 1204 of the cluster chain 1202 points to a correspondingentry 1208 in the discardable FAT 1201, as shown in FIG. 12 b. Asdescribed above with respect to FIGS. 7 and 8, for each file, eachcluster in the cluster chain 1202 within the discardable FAT 1201 pointsto a next sequential cluster of the file until a value such as 1FFFFFFF, as shown in entry 1210, indicates the file's EOF.

It should be appreciated that one cluster chain 1202 may be associatedwith more than one file. For example, as shown in FIG. 12 b, clusterchain 1202 includes a first set of clusters from cluster #6 (element1208) to cluster #9 (element 1210) for a first file 1212 and includes asecond set of cluster from cluster #10 to cluster #11 for a second file1214.

Additionally, it should be appreciated that a primary FAT 1200 andcorresponding discardable FAT 1201 may include more than one clusterchain. For example, as shown in FIGS. 12 a and 12 b, a primary FAT mayinclude the cluster chain 1202 of cluster #6 to cluster #11 and mayinclude a second cluster chain 1216 of cluster #20 to cluster #22.

In other implementations, rather than using a primary FAT 1200 and adiscardable FAT 1201, a file system may utilize a primary FAT 1200 toassociate one or more files with a cluster chain, as described above,and a database or one or more separate location files in place of thediscardable FAT to store physical locations of the one or morediscardable files associated with the cluster chain. The database orlocation files may be text files or binary files that are stored in thenon-discardable area of the file system.

FIG. 13 is a method for managing a storage device using a primary FATand a discardable FAT. FIG. 13 will be described in association withFIG. 1. At step 1310, host 140 receives a request to store file 142 instorage device 100. In some implementations, the storage allocator 144derives the request to store file 142 in the storage device 100 based onone or more write requests associated with the file.

At step 1320, the storage allocator 144 marks the file as “discardable”or as “non-discardable” in a file system structure associated with thestorage device 100 as described above. At step 1320, the file is markedalso in the sense that a discarding priority level is assigned to thefile.

At step 1330, when the file is a discardable file, the storage allocator144 updates a primary FAT to associate a cluster chain that is allocatedto the file with the file. At step 1340, the storage allocator 144updates a discardable FAT to reflect a physical location of the file inthe storage device 100. At step 1350, the storage allocator 144 managesthe storage area 110 of the storage device 100 (through communicationwith the storage controller 120) or manages files that are stored in thestorage area 110 based on the marked file and in accordance with thediscardable FAT. The management of the storage area is similar to thatdescribed above with respect to FIG. 5.

FIG. 14 is a method for managing a storage device using a FAT and adatabase. FIG. 14 will be described in association with FIG. 1. At step1410, host 140 receives a request to store file 142 in storage device100. At step 1420, the storage allocator 144 marks the file as“discardable” or as “non-discardable” in a file system structureassociated with the storage device 100 as described above. At step 1420,the file is marked also in the sense that a discarding priority level isassigned to the file.

At step 1430, when the file is a discardable file, the storage allocator144 updates a FAT to associate a cluster chain that is allocated to thefile with the file. At step 1440, the storage allocator 144 updates adatabase to reflect a physical location of the file in the storagedevice 100. At step 1450, the storage allocator 144 manages the storagearea 110 of the storage device 100 (through communication with thestorage controller 120) or manages files that are stored in the storagearea 110 based on the FAT and the database.

FIG. 15 is a method for managing a storage device using a FAT and alocation file. FIG. 15 will be described in association with FIG. 1. Atstep 1510, host 140 receives a request to store file 142 in storagedevice 100. At step 1520, the storage allocator 144 marks the file as“discardable” or as “non-discardable” in a file system structureassociated with the storage device 100 as described above. At step 1520,the file is marked also in the sense that a discarding priority level isassigned to the file.

At step 1530, when the file is a discardable file, the storage allocator144 updates a FAT to associate a cluster chain that is allocated to thefile with the file. At step 1540, the storage allocator 144 updates alocation file to reflect a physical location of the file in the storagedevice 100. At step 1550, the storage allocator 144 manages the storagearea 110 of the storage device 100 (through communication with thestorage controller 120) or manages files that are stored in the storagearea 110 based on the FAT and the location files.

In yet other implementations, to enhance security, and to prevent thefile system from being destroyed or compromised by file system integrityutilities such as dosfsck (also known as fsck.vfat) or chkdsk, thestorage allocator 144 does not allocate clusters to cluster chainssequentially in the discardable file area to ensure that cluster chainscannot be reconstructed without reading a discardable FAT, database, orone or more location files which store the physical location of adiscardable file. Additionally, range files are generated in the FATthat are associated with one or more of the scrambled clusters of thecluster chain so that utilities such as dosfsck will not turndiscardable files into non-discardable files or reset the flag in theupper bits of the file indicating that the file is discardable. In someimplementations, an attribute such as a hidden, system, directory, orvolume attribute may be enabled that is associated with a range file toprevent a host operating system from accessing the range files.

FIG. 16 is a chart representing a FAT including a cluster chain in whichan order of two or more clusters that comprise the cluster chain havebeen scrambled. As shown in FIG. 16, the clusters that comprise acluster chain that starts at entry 1602 are not contiguous. For example,the order of the cluster chain starting at entry 1602 is cluster #13,cluster #9, cluster #7, cluster #18, and cluster #21. In the FAT, thevalue of each cluster points to the next cluster in the cluster chain,as described above with respect to FIGS. 7 and 8.

In addition to scrambling the order of the clusters that comprise acluster chain associated with one or more files, one or more range filesmay be created in the FAT that comprise one or more clusters of thecluster chain that is associated with the file. In some implementations,each range file may represent all clusters within a range of clustersthat are part of a cluster chain. Due to the association between therange files and the clusters that comprise the cluster chain, utilitiessuch as chkdsk or fsck.vfat will not turn the discardable file intonon-discardable files and FAT recovery utilities will not reset theflags in a FAT32 entry indication that the file is a discardable file.

FIG. 17 is a chart illustrating one or more range files that are createdin the FAT, that each stores at least one cluster of the cluster chainstarting at entry 1602. For example, a first range file 1604 storescluster #7 and cluster #9 from the cluster chain starting at entry 1602,and a second range file 1606 stores cluster #13, cluster #18, andcluster #21 from the cluster chain starting at entry 1602.

A range file may store clusters from more than one cluster chain. Forexample, in addition to the clusters listed above from the cluster chainstarting at entry 1602, the first range file 1604 may store cluster #5and cluster #10 from the cluster chain starting at entry 1608.Similarly, in addition to the clusters listed above from the clusterchain starting at entry 1602, the second range file 1606 may storagecluster #16, cluster #17, and cluster #22 from the cluster chainstarting at entry 1608.

FIG. 18 is a method for managing a storage device using a FAT in whichan order of two or more clusters that comprise a cluster chain isscrambled. FIG. 18 will be described in association with FIG. 1. At step1810, host 140 receives a request to store file 142 in storage device100. At step 1820, the storage allocator 144 marks the file as“discardable” or as “non-discardable” in a file system structureassociated with the storage device 100 as described above. At step 1820,the file is marked also in the sense that a discarding priority level isassigned to the file.

At step 1830, when the file is a discardable file, the storage allocator144 updates a FAT to associate a cluster chain that is allocated to thefile with the file. At step 1840, an order of two or more clusters ofthe cluster chain that are associated with the file are scrambled withinthe FAT based on factors such as an amount of memory within the storagedevice 100, a total size of a cluster chain, a number of clustersbetween two sequential clusters of a cluster chain, and/or flash memorymanagement algorithms that may consider an erase block size, a physicalblock address of each logical address in an allocated block, and/or wearleveling data for each page associated with a physical block address. Insome implementation the order of two or more clusters of the clusterchain are scrambled using a pseudo-random number generator or entropicrandom number generator, which provides an offset within a range foreach cluster that has not been previously allocated. In otherimplementations, the order of two or more clusters of the cluster chainis scrambled using a one-way hash function that takes into accountnon-deterministic values from the host system 140 and/or the storagedevice 100.

At step 1850, a first range file is created in the FAT that comprises atleast one cluster of the cluster chain that is associated with the firstfile. At step 1860, the storage allocator 144 manages the storage area110 of the storage device 100 (through communication with the storagecontroller 120) or manages files that are stored in the storage area 110based on the FAT and the range files.

In yet other implementations, the file system may implement conversionlocks to ensure that a discardable file is not converted to anon-discardable file while the discardable file is open. A discardablefile may be open, for example, during a period of time while thediscardable file is being downloaded to the storage device 100 or duringa period of time before data associated with discardable file is to bereleased to the public, such as when the discardable file is downloadedto the storage device 100 before a release date associated with a movie,song, or program that is associated with the discardable file.Generally, the conversions locks operate such that a discardable filecannot be converted to a non-discardable file when the conversion lockis set.

FIG. 19 is a method for utilizing conversion locks to prevent aconversion of a discardable file when the discardable file is open in afile system implementing a primary FAT and a discardable FAT. FIG. 19will be described in association with FIG. 1. At step 1910, the storageallocator 144 receives a request to convert a discardable file to anon-discardable file. At step 1920, the storage allocator 144 identifiesa value of a conversion lock identifier associated with the discardablefile. At step 1930, the storage allocator 144 determines whether thediscardable file may be converted to a non-discardable file based on thevalue of the conversion lock identifier. Typically, the storageallocator 144 will determine that the discardable file may not beconverted when the value of the conversion lock identifier indicatesthat the discardable file is open and the storage allocator 144 willdetermine that the discardable file may be converted when the value ofthe conversion lock identifier indicates that the discardable file isnot open.

If the storage allocator 144 determines at step 1930 that thediscardable file may not be converted to a non-discardable file, thestorage allocator 144 prohibits the marking of the discardable file asnon-discardable at step 1940. However, if the storage allocator 144determines at step 1930 that the discardable file may be converted to anon-discardable file, the storage allocator 144 proceeds to mark thefile as a non-discardable file in the file system structure associatedwith the storage device 100 at step 1950; update the primary FAT toreflect a physical location of the file at step 1960; and to update thediscardable FAT to remove the physical location of the file at step1970.

It will be appreciated that similar methods are implemented with aconversion lock when a database or location file are used with a primaryFAT in place of the discardable FAT as described above.

In some implementations, an application may be permitted to performoperations such as converting a discardable file to a non-discardablefile, or checking a value of a conversion lock identifier, based on anidentifier associated with the application. Typically, an applicationthat creates or downloads a discardable file may associate a userIDENTIFIER (ID) with the discardable file. The user ID may be an owneruser ID that identifies the application or user that created thediscardable file. In some implementations, the owner user ID is a 4-bytevalue.

The file system 160 provides the owner user ID the ability to definewhat additional user IDs, associated with other users or applications,may access the discardable file and what actions the additional user IDsmay take with respect to the discardable file. It will be appreciatedthat depending on the use of the discardable file, an additional user IDmay be associated with a single application or a single user, or theadditional user ID may be a shared user ID that is associated withmultiple applications or multiple users.

In some implementations, the owner user ID may allow an applicationassociated with an additional user ID to access preview data associatedwith the discardable file. The preview data may be part of thediscardable file where in other implementations the preview data isdistinct from, but associated with, the discardable file. In someexemplary implementations, a discardable file may be a movie and previewdata may include a movie trailer associated with the movie; adiscardable file may be a television program and preview data mayinclude a portion of the television program; a discardable file may bemusic data and preview data may include a portion of the music data; ora discardable file may be a software program and preview data mayinclude a demo version of the software program. In other exemplaryimplementations, preview data may be utilized such that before a releasedate associated with a discardable file the discardable file may not beaccessed but the preview data associated with the discardable file maybe accessed, and then after the release date, both the discardable fileand the preview data may be accessed. In another example, the owner userID may allow an application associated with an additional user ID towrite to a discardable file based on a user ID associated with thediscardable file.

In some implementations, the file system may provide permission bitmasks for the owner user ID to define what operations applicationsassociated with an additional user ID may perform with respect to adiscardable file. One example of permission bit masks for typical usagescenarios is shown in FIG. 20. However, it should be appreciated thatthe owner user ID can override the permissions shown in FIG. 20 andassign any permission to additional user IDs.

Referring to the permissions shown in FIG. 20, an application with aproperties write permission bit 2002 set may modify attributes such asenabling or disabling a conversion lock, setting a time stamp, orwriting a consumption intent universal resource indicator (“URI”) and anapplication with a properties read permission bit 2004 set may readattributes such as a conversion lock, a time stamp, or a consumptionintent URI. An application with a priority permission bit 2006 set canmodify a priority level of a discardable file. An application with apreview read permission bit 2008 set can read preview data associatedwith a discardable file and an application with a preview writepermission bit 2010 set can write preview data associated with adiscardable file. An application with a read permission bit 2012 set mayread a discardable file and an application with a write permission bit2014 set may write to a discardable file. Typically, only an applicationassociated with an owner user ID that is associated with a discardablefile will have these permissions. An application with a convertpermission bit 2016 set can convert a discardable file to anon-discardable file.

It is noted that the methodology disclosed herein, of marking files andassigning to them discarding levels in associated file system, may havemany useful applications, one of which is restoring a storage usagesafety margin to guarantee sufficient storage space for user files. Forexample, a discarding level assigned to a file may be used to remap fileclusters to a lower-performing flash module, or to clear the clustersupon request.

The articles “a” and “an” are used herein to refer to, one or to morethan one (i.e., to at least one) of the grammatical object of thearticle, depending on the context. By way of example, depending on thecontext, “an element” can mean one element or more than one element. Theterm “including” is used herein to mean, and is used interchangeablywith, the phrase “including but not limited to”. The terms “or” and“and” are used herein to mean, and are used interchangeably with, theterm “and/or,” unless context clearly indicates otherwise. The term“such as” is used herein to mean, and is used interchangeably, with thephrase “such as but not limited to”.

Having thus described exemplary embodiments of the invention, it will beapparent to those skilled in the art that modifications of the disclosedembodiments will be within the scope of the invention. Alternativeembodiments may, accordingly, include more modules, fewer modules and/orfunctionally equivalent modules. The present disclosure is relevant tovarious types of mass storage devices such as SD-driven flash memorycards, flash storage devices, non-flash storage devices, “Disk-on-Key”devices that are provided with a Universal Serial Bus (“USB”) interface,USB Flash Drives (“UFDs”), MultiMedia Card (“MMC”), Secure Digital(“SD”), miniSD, and microSD, and so on. Hence the scope of the claimsthat follow is not limited by the disclosure herein.

What is claimed is:
 1. A method for managing operations associated witha discardable file, the method comprising: in a host to which a storagedevice is operatively coupled: associating a discardable file with anowner user ID of a first application that creates or downloads thediscardable file, wherein the discardable file comprises unsolicitedcontent that has not been requested by a user of the storage device andwhere the discardable file comprises a file that has been marked asdiscardable in a file system structure associated with the storagedevice and has been assigned a discarding priority level in advance ofstoring the discardable file; defining, with the first applicationassociated with the owner user ID, a set of permissions for anadditional user ID that is associated with the discardable file, wherethe additional user ID is associated with a second application and theset of permissions define whether the second application may access thediscardable file and one or more actions that the second application maytake with respect to the discardable file; receiving, from the secondapplication associated with the additional user ID, a request to performa conversion operation associated with the discardable file to convertthe discardable file to a user file, where a user file is a file that auser of the storage device willingly stores in the storage device;determining whether the second application associated with theadditional user ID may perform the conversion operation based on the setof permissions; and managing the conversion operation to convert thediscardable file to a user file based on the determination.
 2. Themethod of claim 1, wherein the first application associated with theowner user ID downloads the discardable file to the storage device. 3.The method of claim 1, wherein the operation comprises: modifying anattribute associated with the discardable file.
 4. The method of claim3, wherein the attribute comprises at least one of a conversion lockidentifier, a time stamp, a consumption intent universal resourceindicator, and a priority level.
 5. The method of claim 1, wherein theoperation comprises: reading an attribute associated with thediscardable file.
 6. The method of claim 5, wherein the attributecomprises at least one of a conversion lock identifier, a time stamp, aconsumption intent universal resource indicator, and a priority level.7. The method of claim 1, wherein the discardable file comprises previewdata.
 8. The method of claim 1, wherein the discardable file is distinctfrom preview data.
 9. The method of claim 1, wherein the additional userID is a shared user ID associated with multiple users.
 10. The method ofclaim 1, wherein the additional user ID is a shared user ID associatedwith multiple applications.
 11. The method of claim 1, wherein managingthe conversion operation to convert the discardable file to a user filebased on the determination comprises: prohibiting the second applicationassociated with the additional user ID from converting the discardablefile to a user file.
 12. The method of claim 1, wherein managing theconversion operation to convert the discardable file to a user filebased on the determination comprises: allowing the second applicationassociated with the additional user ID to convert the discardable fileto a user file.
 13. A storage system comprising: a communicationinterface; and a storage allocator for managing a file system associatedwith a storage device, the storage allocator including a processor formanaging operations associated with a discardable file that is stored inthe storage device; wherein the processor is configured to: associatethe discardable file with an owner user ID of a first application thatcreates or downloads the discardable file, wherein the discardable filecomprises unsolicited content that has not been requested by a user ofthe storage device and where the discardable file comprises a file thathas been marked as discardable in a file system structure associatedwith the storage device and has been assigned a discarding prioritylevel in advance of storing the discardable file; define, with the firstapplication associated with the owner user ID, a set of permissions foran additional user ID that is associated with the discardable file,where the additional user ID is associated with a second application andthe set of permissions define whether the second application may accessthe discardable file and one or more actions that the second applicationmay take with respect to the discardable; receive, from the secondapplication associated with the additional user ID, a request over thecommunication interface to perform a conversion operation associatedwith the discardable file to convert the discardable file to a userfile, where a user file is a file that a user of the storage devicewillingly stores in the storage device; determine whether the secondapplication associated with the additional user ID may perform theconversion operation based on the set of permissions; and manage theconversion operation to convert the discardable file to a user filebased on the determination.
 14. The storage system of claim 13, whereinto manage the operation associated with the discardable file, theprocessor is configured to allow the second application associated withthe additional user ID to read preview data associated with thediscardable file.
 15. The storage system of claim 13, wherein theadditional user ID is a shared user ID associated with multiple users.16. The storage system of claim 13, wherein the additional user ID is ashared user ID associated with multiple applications.
 17. The method ofclaim 1, where the conversion operation to convert the discardable fileto a user file comprises storing the converted user file in the storagedevice as a user file.
 18. The method of claim 13, wherein to manage theconversion operation, the processor is further configured to store theconverted user file in the storage device as a user file.